ACE+TAO Opensource Programming Notes/A more useful client and server example

In the previous client server example, 4 files were edited to present a really basic example of a CORBA enabled application which presented a RMI interface in a factory type interface. Not terribly interesting, or useful. Ideally, one of these applications would act to serve up networked objects/entities which could answer questions as well as perform tasks. It would also be nice to have it perform these tasks with a database back end. The point of such a service would be to implement an in-memory database, the members of which are capable of individually acting like instantiated objects.

CORBA provides a framework to hang networked objects on called the Portable Object Adapter (POA). This framework, and a rather complicated framework it is, is capable of keeping references to your objects, performing reference counting of your objects, and doing garbage collection on spent objects. For this example, we'll only use the POA to store references to the factory object. Other services the POA offers we'll ignore for the moment, and perform our own object lifetime operations. In later examples, we'll explore the utility of using more of the features of a POA, however, since these features aren't generated by the IDL compiler and require knowledge about the specific ORB you'll be using (in this case TAO), we'll use this simpler example first, and build on it later.

This example was originally used to proof out a performance improvement for some rather inefficient code which was originally written in Java and running under a Tomcat instance. My job was to get more bank for the customer's buck out of their server, and incidentally, solve some of their garbage collection issues they were having in Tomcat. My solution was to factor their existing transient in-memory entities out of Tomcat, and into an in-memory CORBA server. The server you're about to see certainly isn't the most efficient design, however, it is designed to mimic exactly the customer's design. The point of this was to demonstrate the performance improvements available to the JAVA designers in as recognizable a fashion as possible. Following the initial demonstration, and first flush of embarrassment, subsequent improvements were added on to further flush out the available features.

Design

This client's issue was with a customer framework that was chewing up too much resources, freezing when GC operations were in effect, and generally not scaling as well as they'd hoped. To proof the improvements they could hope to realize, I went out onto the net, and downloaded one of those free, randomly generated address/credit card databases. I then used this to fill an in-memory DBMS based in Boost's Muti-Index Container, and then used a CORBA factory idiom to feed these objects to clients. When the clients were done with these objects, they called a destroy method on them and the server's POA would clean them up. Simple to understand, but not the speediest design.

The IDL

Source File Actor.idl

// Forward declare
interface Actor;
struct Person{
  unsigned long id;
  string fname;
  string mname;
  string lname;
  unsigned long address;
};

// = TITLE
//   A factory class for Actor interfaces
interface My_Factory
{
  typedef unsigned long uint;
  typedef sequence<uint> People;
  //Get an interface to an actor via its ID
  Actor get_actor (in uint actor_id);
  //Get one or more People by searching on the first name. Returns a sequence of actor_ids.
  People get_sloppy_match_fname(in string fname);
  //Get one or more People by searching on the first and last names, returns a sequence of actor_ids.
  People get_sloppy_match_flname(in string fname, in string lname);
  uint add_actor(in Person person); //Add a person to the in-memory db and the backing store, return person_id
  uint delete_actor(in uint actor_id); //Delete from in-memory and backing store, associated addresses will likely be cascaded on the backend.
};

interface Actor
{
  readonly attribute Person person;
  typedef unsigned long uint;
  typedef sequence<uint> Addresses;
  uint get_identity(); 
  string get_fname();
  string get_mname();
  string get_lname();
  Addresses get_addresses();
  void setPersonId(in uint ps);
  void destroy();
};

Now, run your IDL through the compiler to get your machine generated code out:

tao_idl -GI Actor.idl

This generates a bunch of files, the one we'll be editing first, is the ActorI.h file. This is where the interface for the IDL is stored. Since we will be needing a database connection in the factory class to perform some operations, our first edit will be to add a database connection object to the My_Factory_i class. Just put it up near the top, and we'll initialize it when the factory starts up. For this example, I used PostgreSQL as the database. An example of the code edit I made is listed below. That's it for this file.

The Interface

Source File ActorI.h

/*! \class My_Factory_i
*   \brief Factory class for creating instances of Actor and Location
*
 *   Code generated by The ACE ORB (TAO) IDL Compiler v1.8.2
*/
class  My_Factory_i
  : public virtual POA_My_Factory
{
public:
  /*! \var pqxx::connection *c
  *   \brief Postgres Connectin variable
  *
  *   This variable gets used throughout the class for the duration of its life
  *   as the variable responsible for enabling writes back to the database
  */
  pqxx::connection *c;
...

Now on to the ActorI.cpp file. The implementation is jest a matter of filling out the IDL created carcase. This source needs to create a live database connection, connect to the multi-index in-memory database, perform searches, and write certain things back to the database. The most difficult thing to figure out how to do is how to manage the created CORBA objects. The documentation from the open-source support community is pretty thin on the ground, and generally of a type not terribly useful for development. There is however a book for sale by OCI which helps in presenting useful examples which are great for development. This example was inspired by Henning's book on using CORBA in combination with the STL containers. So, here's the interface file.

Source File Actor.cpp

#include "ActorI.h"

// Implementation skeleton constructor
My_Factory_i::My_Factory_i (void)
{
  c = new pqxx::connection("dbname=ecarew user=ecarew");
}

// Implementation skeleton destructor
My_Factory_i::~My_Factory_i (void)
{
}

::Actor_ptr My_Factory_i::get_actor (
  ::My_Factory::uint actor_id)
{
  Actor_i *actorInterface = new ::Actor_i();
  PortableServer::ServantBase_var actor_safe (actorInterface);	// NEW
  actorInterface->setPersonId(actor_id);
  return actorInterface->_this();
}

::My_Factory::People * My_Factory_i::get_sloppy_match_fname (
  const char * fname)
{
  pqxx::work w(*c);
  std::string sql = "SELECT id from persons where fname = '";
  sql += fname;
  sql += "'";
  pqxx::result r = w.exec(sql);
  ::My_Factory::People* Pseq = new ::My_Factory::People(r.size());
  Pseq->length(r.size());
  int rnum = 0;
  for(pqxx::result::const_iterator row = r.begin(); row != r.end(); ++row){
    int id = 0;
    row[0].to(id);
    (*Pseq)[rnum++] = id;
  }
  return Pseq;
}

::My_Factory::People * My_Factory_i::get_sloppy_match_flname (
  const char * fname,
  const char * lname)
{
  pqxx::work w(*c);
  std::string sql = "SELECT id from persons where fname = '";
  sql += fname;
  sql += "' and lname = '";
  sql += lname;
  sql += "'";
  pqxx::result r = w.exec(sql);
  ::My_Factory::People*  Pseq = new ::My_Factory::People(r.size());
  Pseq->length(r.size());
  int rnum = 0;
  for(pqxx::result::const_iterator row = r.begin(); row != r.end(); ++row){
    int id = 0;
    row[0].to(id);
    (*Pseq)[rnum++] = id;
  }
  return Pseq;
}

::My_Factory::uint My_Factory_i::add_actor (
  const ::Person & person)
{
  _person iTarget(pers.size() - 1, std::string(person.fname), std::string(person.mname), std::string(person.lname), 0);
  pers.insert(iTarget);
  return pers.size();
}

::My_Factory::uint My_Factory_i::delete_actor (
  ::My_Factory::uint actor_id)
{
  person_set::nth_index<0>::type& person_index = pers.get<0>();
  person_index.erase(_person(actor_id, "", "", "", 0));
  pers.insert(_person(actor_id, "", "", "", 0));
  return actor_id;
}

// Implementation skeleton constructor
Actor_i::Actor_i (void)
{
  ps = new ::Person;
}

// Implementation skeleton destructor
Actor_i::~Actor_i (void)
{
}

::Person * Actor_i::person (void)
{
  return ps._retn();
}

::Actor::uint Actor_i::get_identity (void)
{
  return ps->id;
}

char * Actor_i::get_fname (void)
{
  return CORBA::string_dup(ps->fname);
}

char * Actor_i::get_mname (void)
{
  return CORBA::string_dup(ps->mname);
}

char * Actor_i::get_lname (void)
{
  return CORBA::string_dup(ps->lname);
}

::Actor::Addresses * Actor_i::get_addresses (void)
{
  // Add your implementation here
}

void Actor_i::setPersonId (
  ::Actor::uint ps)
{
  person_set::nth_index<0>::type::iterator 
      it=pers.get<0>().find(_person((unsigned int)pss, "", "", "", 0));
  ps->id = (*it).id;
  ps->fname = (*it).fname.c_str();
  ps->mname = (*it).mi.c_str();
  ps->lname = (*it).lname.c_str();
  ps->address = (*it).address;
}

void Actor_i::destroy (void)
{
  // Get the id of this servant and deactivate it from the default POA.
  PortableServer::POA_var poa = this->_default_POA();
  PortableServer::ObjectId_var oid = poa->servant_to_id(this);
  poa->deactivate_object(oid);
}

Let's go over the code. The constructor for the factory creates the postgres connection object. The factory destructor doesn't do anything. The factory get_actor member does the job of instantiating a new Actor_i interface object. Figuring out how to write this code was quite a job for me. Since I didn't have a copy of the OCI document, I was left using more generic documents such as Henning's book. While these books will give you an idea of how something can be architected, specific implementations are definitely on you as the developer. The problem with this code as shown in the getter is that an extra reference count is placed on this object as it is created. I can't find any way to decrement this, so the effect is that when we are done with the object in our client, it gets decremented once, but not enough to get the object garbage collected. To fix this, the created object has a built in destructor function we will be discussing later. That's it for the moment on the factory.

The actor class accesses the in-memory database with the Person object. you can see it created in the constructor for Actor_i.

Source file ActorI.h

class  Actor_i
  : public virtual POA_Actor
{
public:
  Person_var ps;
  // Constructor

The Application

The in-memory database is defined in the header file person.h, and, as mentioned earlier, is based on Boost's multi-index class. This allows the designer to create a container of objects which can have multiple indexes in them. The file is as follows:

Source File person.h

/***************************************************************************
 *   Copyright (C) 2010 by Evan Carew   *
 *                                                                         *
 *   This program is free software; you can redistribute it and/or modify  *
 *   it under the terms of the GNU General Public License as published by  *
 *   the Free Software Foundation; either version 2 of the License, or     *
 *   (at your option) any later version.                                   *
 *                                                                         *
 ***************************************************************************/
#ifndef PERSON_H
#define PERSON_H
#include <boost/multi_index_container.hpp>
#include <boost/multi_index/ordered_index.hpp>
#include <boost/multi_index/identity.hpp>
#include <boost/multi_index/member.hpp>
struct _person
{
  int         id;
  std::string fname;
  std::string mi;
  std::string lname;
  int address;

  _person(int id,const std::string& fname, const std::string mi, const std::string lname, int address):id(id),fname(fname), mi(mi), lname(lname), address(address){}
  
  _person(const _person&p){id = p.id; fname=p.fname; mi=p.mi; lname=p.lname; address=p.address;}

  bool operator<(const _person& p)const{return id<p.id;}
};

// define a multiply indexed set with indices by id and name
typedef boost::multi_index_container< _person, boost::multi_index::indexed_by<    // sort by employee::operator<
    boost::multi_index::ordered_unique<boost::multi_index::identity<_person> >,
          // sort by less<string> on name
    boost::multi_index::ordered_non_unique<boost::multi_index::member<_person,std::string,&_person::lname> >
        > > person_set;

extern person_set pers;
#endif

If you look in ActorI.cpp, you'll see in void Actor_i::setPersonId (::Actor::uint ps) how the multi-index object is used to search for data. To get an idea for how this code works, with respect to the Boost container, have a look at their documentation of Boost::Multi-index the library.

Now, on to the server code. This code is rather simple, and mostly driven by boilerplate code. The IDL compiler doesn't generate any of this, tho it probably could. in this case, the server creates an ORB, attaches to the naming service, and initializes the in-memory database from the postgreSQL database.

Source File objectserver.cpp

/***************************************************************************
 *   Copyright (C) 2010 by Evan Carew   *
 *                                                                         *
 *   This program is free software; you can redistribute it and/or modify  *
 *   it under the terms of the GNU General Public License as published by  *
 *   the Free Software Foundation; either version 2 of the License, or     *
 *   (at your option) any later version.                                   *
 *                                                                         *
 ***************************************************************************/

#ifdef HAVE_CONFIG_H
#include <config.h>
#endif

#include <iostream>
#include <cstdlib>
#include <string>
#include <pqxx/pqxx>
#include <ace/streams.h>
#include <tao/PortableServer/PortableServer.h>
#include <orbsvcs/CosNamingC.h>
#include "ActorI.h"
#include "ActorS.h"
#include "person.h"

using namespace std;
using namespace pqxx;
using namespace ::boost;
using namespace ::boost::multi_index;
using namespace CORBA;

person_set pers;

int main(int argc, char *argv[])
{
  connection c("dbname=ecarew user=ecarew");
  work w(c);
  int id, address;
  string fname, lname, mi;
  
  extern person_set pers;
  
  result r = w.exec("SELECT id, fname, mi, lanem, addr from persons");
  for(pqxx::result::const_iterator row = r.begin(); row != r.end(); ++row){
    row[0].to(id);
    row[1].to(fname);
    row[2].to(mi);
    row[3].to(lname);
    row[4].to(address);
    pers.insert(_person(id, fname, mi, lname, address));
  }
  w.commit();
  
  // First initialize the ORB, that will remove some arguments...
  ORB_var orb = ORB_init (argc, argv);

  // Get a reference to the RootPOA
  Object_var poa_object = orb->resolve_initial_references ("RootPOA");
  // narrow down to the correct reference
  PortableServer::POA_var poa = PortableServer::POA::_narrow (poa_object.in ());
  // Set a POA Manager
  PortableServer::POAManager_var poa_manager = poa->the_POAManager ();
  // Activate the POA Manager
  poa_manager->activate ();
// Create the servant
  My_Factory_i my_factory_i;
 
  // Activate it to obtain the object reference
  My_Factory_var my_factory = my_factory_i._this ();
 
  // Put the object reference as an IOR string
  char* ior = orb->object_to_string (my_factory.in ());
  //Now we need a reference to the naming service
  Object_var naming_context_object = orb->resolve_initial_references ("NameService");
  CosNaming::NamingContext_var naming_context = CosNaming::NamingContext::_narrow (naming_context_object.in ());
  //Then initialize a naming context
  CosNaming::Name name (1);
  name.length (1);
  //store the name in the key field...
  name[0].id = string_dup ("Actors");
  //Then register the context with the nameing service
  naming_context->rebind (name, my_factory.in ());
  //By default, the following doesn't return unless there is an error
  orb->run ();
 
  // Destroy the POA, waiting until the destruction terminates
  poa->destroy (1, 1);
  orb->destroy ();

  return EXIT_SUCCESS;
}

Now that we've discussed all the code required to make the server, it's worth noting some of the parts of the application which make it worthwhile as a useful example. note for instance that some of the methods in the factory as well as the entity class, Actor, return complex types. Few, if any of the on-line examples I found discussed how to do this. The main reason many of the other examples don't show this is its a fairly deep topic which requires the developer to first understand how the CORBA spec deals with its own funky brand of memory management. I won't go into all the options here, except to say that most applications I've worked on have either used integral types, in which case, there are no concerns, or a fairly simple usage of the CORBA containers such as sequence, string, and the odd struct. The previously described code uses all of them and shows how to deal with the memory. The key is in name of the automatically generated variable names which come out of the IDL compiler. Those names ending in _var are CORBA's idea of an STL::auto_ptr. This means that they own the pointed to object, and like a typical stl container, when they go out of scope, they clean up their own memory, and that contained in the client and server ORBs. Strings merrit special mention as they seem to have their own separate treatment. Notice that in the code, instead of using straight assignment, or new, or some other special operator, CORBA uses the string_dup() function. Don't think about it, just do it. It's required by the spec. As for the sequence and struct, using the _var variable is sufficient.

The Client

Most of the client is boilerplate code, i.e., setup of the CORBA connection. The actual usage of the CORBA object looks just like a function call. As mentioned above, the only thing to watch out for is the usage of the three object types struct, sequence, and string. Have a look at the example client below to get a feel for how it works:

Source objectclient.cpp

/***************************************************************************
 *   Copyright (C) 2010 by Evan Carew                                      *
 *                                                                         *
 *   This program is free software; you can redistribute it and/or modify  *
 *   it under the terms of the GNU General Public License as published by  *
 *   the Free Software Foundation; either version 2 of the License, or     *
 *   (at your option) any later version.                                   *
 ***************************************************************************/
#ifdef HAVE_CONFIG_H
#include <config.h>
#endif

#include <iostream>
#include <cstdlib>
#include <string>
#include <ace/streams.h>
#include <tao/PortableServer/PortableServer.h>
#include <orbsvcs/CosNamingC.h>
#include <sys/time.h>
#include "ActorI.h"
#include "ActorC.h"
#include "person.h"

using namespace std;
using namespace ::boost;
using namespace CORBA;
person_set pers;

int main(int argc, char **argv){
  struct itimerval t_new, t_old, t_result;
  // First initialize the ORB, that will remove some arguments...
  ORB_var orb = ORB_init (argc, argv);

  // Obtain Naming Service reference.
  Object_var naming_context_object = orb->resolve_initial_references ("NameService");
  CosNaming::NamingContext_var naming_context = 
      CosNaming::NamingContext::_narrow (naming_context_object.in ());

  CosNaming::Name name (1);
  name.length (1);
  name[0].id = string_dup ("Actors");

  Object_var factory_object =  naming_context->resolve (name);

  // Now downcast the object reference to the appropriate type
  My_Factory_var factory = My_Factory::_narrow (factory_object.in ());
  t_new.it_interval.tv_sec = 1;
  t_new.it_interval.tv_usec = 0;
  t_new.it_value.tv_sec = 1;
  t_new.it_value.tv_usec = 0;
  setitimer(ITIMER_PROF, &t_new, &t_old);
  // Get the person object
  for(int i = 1; i < 1000; i++){
    Actor_var actor = factory->get_actor (i);
    string actor_fname = actor->get_fname();
    //cout << actor_fname << endl;
    actor->destroy();
  }
  getitimer(ITIMER_PROF, &t_result);
  
  orb->shutdown(1);
  orb->destroy();
  //cout <<  1000000 - t_result.it_value.tv_usec << " microseconds"
  //    << endl;
 
  return 0;
}

Deploying the App

Compilation

The TAO web site does a passable job of helping the developer new to ACE+TAO through the task of compiling their new applications. Despite these instructions, there are often some questions on their list serve regarding apps not compiling or linking. The compilation command you are going to need with a 5.8.x or later version of ACE+TAO is likely to require the following local objects:

ActorC.cpp ActorI.cpp ActorS.cpp objectserver.cpp

and the link portion of you application is likely to require the following objects:

-lpqxx -lTAO_PortableServer -lTAO_CosNaming -lTAO_AnyTypeCode -lACE -lTAO

Deployment

Until recently, ACE+TAO didn't have a deployment plan worth mentioning. I'm not sure which version it was that they acquired a make install target for their platforms, suffice it to say, you want to use their more modern versions, such as 5.8.x and above. If you are on a redhat style linux distro, they even have pre-compiled RPMs. Installing ACE+TAO will place the libraries in your system's common directory, as well as placing the tao binaries in some common bin location like /usr/bin. Once this prerequisite is over with, installing your app is simplicity itself. just copy your binary to a directory from which you can run it. One more thing, remember that you're probably going to want to have a copy of a name server running somewhere on your net, configured to load at boot time.